Form recognition system, method, program, and storage medium

ABSTRACT

The similarity between a search form and a registered form is calculated on a page-by-page basis in a predetermined order. Every time the page similarity is calculated, it is determined whether or not a value representing the similarity is smaller than a predetermined threshold value. If it is determined that the similarity value is smaller than the threshold value, the registered form is removed, at this point, from candidates for a recognition target. A form having a possibility of page repetition can also be recognized.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to systems, methods, programs, and storagemedia for automatic form recognition. Specifically, the presentinvention relates to a system, method, program, and storage medium forthe recognition of forms, each containing multiple pages.

2. Description of the Related Art

Form recognition, in which forms are automatically classified accordingto previously registered formats, is a highly effective technique forperforming entry processing for large quantities of forms. A formrecognition system extracts feature values of form image data read by ascanner or the like, and creates form format data. The form recognitionsystem then calculates the similarity between a search form and eachregistered form, and determines a registered form with the highestsimilarity to be the recognition result.

For form recognition, Japanese Patent Laid-Open No. 2000-285187 focusesattention on tables appearing on a form and determines the similaritybased on the proportion of each table area to the total area of all thetables, the similarity being close to that determined based on visualappearance. However, since the similarity is calculated for each imageentered, that is, on a page-by-page basis, the form recognition systemdisclosed in Japanese Patent Laid-Open No. 2000-285187 is unsuitable forthe recognition of forms, each containing multiple pages.

For the recognition of forms with multiple pages, Japanese PatentLaid-Open No. 10-269311 discloses a method in which a partition form isplaced at the top of each block of pages in order to process multiplepages as a single unit. That is, it is determined, every time a form isread, whether or not the form is a partition form, and, if it is, thesubsequent form through to the form immediately before the nextpartition form are processed as a single unit. However, the formrecognition disclosed in Japanese Patent Laid-Open No. 10-269311 isdisadvantageous in that it involves a cumbersome process of insertingpartition forms in dealing with forms with multiple pages.

SUMMARY OF THE INVENTION

The present invention has been made to solve such conventional problemsdescribed above, and allows for accurate and efficient recognition ofmultiple pages of forms without requiring a cumbersome process, such asinsertion of partition forms.

In an aspect of the present invention, a form recognition system foridentifying which of a plurality of previously registered formscorresponds to images of a search form with multiple pages includes:feature obtaining means for obtaining page feature values from thesearch form in a predetermined order; similarity calculating means forcalculating a similarity value representing similarity between the pagefeature values obtained from the search form and page feature values ofa registered form corresponding to the page feature values of the searchform; target-form selecting means for removing the registered form fromcandidates for a recognition target if the similarity value is smallerthan a predetermined value; and form determining means for determining aform of recognition result, based on registered forms not removed by thetarget-form selecting means, by repeatedly processing each page of thesearch form using the feature obtaining means, the similaritycalculating means, and the target-form selecting means.

In another aspect of the present invention, a form recognition systemfor identifying which of a plurality of previously registered formscorresponds to images of a search form with multiple pages includes: aregistered form memory for storing the possibility of page repetition ina registered form; feature obtaining means for obtaining feature valuesof each page of the search form; similarity calculating means forcalculating a similarity value representing similarity between thefeature values of each page of the search form obtained by the featureobtaining means, and the feature values of each page of the registeredform; and form determining means for determining a registered formcorresponding to the search form, based on the similarity calculated bythe similarity calculating means. The similarity calculating means alsocalculates, if there is a possibility of page repetition in a registeredform, the similarity of a page having the possibility of being repeated.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing a form recognition system of a firstembodiment according to the present invention.

FIG. 2 is a flowchart showing registration processing in the formrecognition system in FIG. 1.

FIG. 3 is a flowchart showing recognition processing in the formrecognition system in FIG. 1.

FIG. 4 is a table showing the data structure of a registered form in theform recognition system in FIG. 1.

FIG. 5 is a table showing the data structure of a registered form in theform recognition system of a second embodiment according to the presentinvention.

FIG. 6 shows a memory map of a storage medium for storing various dataprocessing programs for the form recognition system in FIG. 1.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

FIG. 1 is a block diagram showing a form recognition system of a firstembodiment according to the present invention.

In FIG. 1, the form recognition system includes an external storage 104,such as a magnetic disk, and an image scanner 108. The form recognitionsystem reads form image data from the external storage 104, or scansforms with the image scanner 108.

The form recognition system includes a CPU 101 for overall control. Acontrol program for the CPU 101 is stored in a ROM (read-only memory)102. Form images entered from the image scanner 108 are held in a RAM(random access memory) 103 and are processed for form recognition,according to the control program stored in the ROM 102. While a computerprogram for implementing the present invention is stored in a ROM in thepresent embodiment, the program may be stored in another storage medium,or may be received from another device via a network or the like.

Feature values for use in form recognition are stored in the ROM 102together with the control program, or are read from the external storage104 into the RAM 103.

Preprocessing, such as noise reduction, is carried out, as necessary,prior to the form recognition.

As shown in FIG. 1, the form recognition system includes a display 105,a keyboard 106, and a pointing device 107, such as a mouse. A networkinterface (IF) 109 is provided for communicating with remote systems(not shown) to send and receive programs, recognition dictionaries,data, and the like.

The present invention can be implemented by a general-purpose computer.In this case, a control program on a medium or the like may be stored inthe external storage 104 to be executed by the CPU 101 under operatorcontrol or the like.

The form recognition system may have a structure in which the imagescanner 108, the display 105, and the like are connected thereto via aninterface.

Processing in the form recognition system in FIG. 1 will now bedescribed.

FIG. 2 is a flowchart showing the processing of registering a form to beused as a template in the form recognition system. FIG. 3 is a flowchartshowing form recognition processing for identifying a registered formcorresponding to a form entered.

In FIG. 2, the processing of form registration is executed according tothe following steps.

Step S201: First, image data of a form to be registered is inputtedthrough the image scanner 108 on a page-by-page basis.

Step S202: Feature values are extracted from a page of the image datainputted in step S201. The feature values include the size, position,and number of table frames in the form, and the number of cells includedin each table frame.

Step S203: It is determined whether or not the extraction of featurevalues in all pages of the form to be registered has been completed. Ifnot, the process returns to step S201. Steps S201 to S203 are repeateduntil the feature value extraction in all pages is completed.

Step S204: The feature values extracted from each page, in step S202,are stored while all the pages are associated with one another into asingle form. The registration of a single form is thus completed. Theimage data and feature values of the registered form are stored in theRAM 103 or in the external storage 104.

The form data registered through the process in FIG. 2 is structured andstored, for example, as a table shown in FIG. 4. More specifically, athree-page form ID1 includes image data A on the first page, image dataB on the second page, and image data C on the third page. A four-pageform ID2 includes image data D on the first page, image data E on thesecond page, image data F on the third page, and image data G on thefourth page. A two-page form ID3 includes image data H on the first pageand image data I on the second page.

Referring now to FIG. 3, the processing of form recognition is executedaccording to the following steps.

Step S301: In the present embodiment, feature values are extracted (stepS305) from data of an image for each page (hereinafter referred to as“search image”) of a form to be identified (hereinafter referred to as“search form”). The feature values of each page are sequentially storedin a buffer. First, in step S301, it is determined whether or notfeature values that have not yet been processed for recognition exist inthe buffer. If it is determined that the feature values exist, theprocess proceeds to step S306. If not, the process proceeds to stepS302.

Step S302: In step S302, it is determined whether or not the entry ofall pages of a search form has been completed. If it is determined thatthe entry of all pages has been completed, the process proceeds to stepS317. If not, the process proceeds to step S303.

Step S303: The entry of search images of a search form starts in stepS303. Then the process proceeds to step S304.

Step S304: Predetermined feature values are extracted from a page of thesearch image entered. As in the case of step S202 for a registered form,the feature values include the size, position, and number of tableframes in the form, and the number of cells included in each tableframe.

Step S305: Then, the feature values extracted in step S304 are stored inthe buffer described with reference to step S301.

Step S306: If, in step S301, feature values have already been stored inthe buffer, the top of the feature values in the buffer are extracted,and the process proceeds to step S307.

Step S307: The similarity between the feature values extracted in stepS306 and those of a corresponding page of each registered form iscalculated. A search image on the first page needs to be compared to thecorresponding pages in all the registered forms, because the registeredforms have not yet been narrowed down at this point. That is, if theregistered forms listed in the table in FIG. 4 are to be compared, thesimilarities to the image data A, D, and H on the first pages of theforms ID1, ID2, and ID3, respectively, are calculated.

Step S308: A value representing the similarity determined in step S307is compared to a predetermined threshold value. If the similarity valueis smaller than the threshold value, the process proceeds to step S309.If the similarity is equal to or above the threshold, the processproceeds to step S310.

Step S309: A registered form with a corresponding page having asimilarity value smaller than the threshold value is removed to narrowdown the recognition targets. The process then proceeds to step S310.If, for example, the similarity value to the image data D is smallerthan the threshold value, the form ID2 is removed to narrow therecognition targets down to the forms ID1 and ID3.

Step S310: It is determined whether or not the entry of search imagesand the calculation of feature values until the last page of the searchform have been completed. If completed, the process proceeds to stepS311. The process proceeds to step S315 if any pages remain unprocessed.

Step S311: It is determined whether or not the number of recognitiontargets has been narrowed down to one. If so, the process proceeds tostep S312. If a plurality of registered forms still remain asrecognition targets, the process proceeds to step S313.

Step S312: The only remaining form is determined to be the result ofform recognition, and the process proceeds to step S314.

Step S313: If a plurality of forms still remain, for example, thesimilarities of all pages of each form are averaged so as to determinethe form with the maximum average value to be the result of formrecognition. The processing of determining the result of formrecognition in step S313 is not limited to the way described above. Theresult of form recognition can be determined using various algorithms.For example, the lowest page similarities are compared to determine theform having a page with the maximum lowest similarity to be the resultof form recognition.

Step S314: If the result of recognition is determined in step S312 or instep S313, feature values are deleted, by the amount corresponding tothe number of pages of the resultant form, from the buffer in which thefeature values are stored, for the subsequent recognition processing.

For example, in step S308, if the image data B and I on the second pagesof the forms ID1 and ID3, respectively, are equal to or above thethreshold value in similarity, and further, if the image data C on thethird page is equal to or above the threshold value in similarity, boththe forms ID1 and ID3 remain as recognition targets. If it is determinedthat the form ID3 is the recognition result, two pages (image data H andI) of feature values are deleted from the buffer.

As for the form ID1, the image data C on the third page has beenpre-read and the feature values have already been extracted and storedin the buffer, since the similarity was to be calculated. Therefore, thefeature values of the image data C can be immediately extracted torestart the recognition processing using the image data C as image datafor the first page.

On the other hand, if it is determined that the form ID1 is the resultof recognition, feature values of all the pages are deleted from thebuffer. Processing then returns to step S301 for subsequent recognitionprocessing. In the subsequent recognition processing, another image willbe entered in step S303.

Step S315: If similarity calculation has not yet been completed to thelast page of the search form, it is determined whether or not aregistered form to be recognized still remains. If there is still aregistered form remaining, the process returns to step S301 and thesubsequent processing is repeated. If it is determined in step S315 thatno registered form to be recognized remains, the process proceeds tostep S316.

Step S316: If no registered form to be recognized remains, therecognition result of the first page of the search form is determined tobe “Unknown”. At the same time, information at the top of the featurevalues stored in the buffer is deleted. Then, a page that was the secondpage of the same search form becomes the first page, and the formrecognition processing starts again by returning to step S301.

Steps S317 and S318: If it is determined in step S302 that the entry ofsearch forms has been completed, it is determined in step S317 whetheror not any feature values remain in the buffer. An empty buffer meansthat the recognition of all search images has been successfullycompleted. However, if any feature values remain in the buffer, the formrecognition is unsuccessful. In step S318, the results of suchunsuccessful recognition are all determined to be “Unknown”. The user isnotified of the presence of “Unknown” search forms.

As described above, in the first embodiment, the recognition of multiplepages of forms with various page counts can be carried out.

After the calculation of feature values of a search image, unsuccessfulrecognition, due to, for example, the absence of sufficiently similarregistered form or the difference in page count, is completed bydetermining the result to be “Unknown.” In such a case, even if thesearch image needs to be processed again, the recalculation of featurevalues is not required.

FIG. 6 shows a memory map of a storage medium for storing various dataprocessing programs executable in the form recognition system. Thememory map includes directory information, a first data-processingprogram including a program code for executing the processing in theflowchart shown in FIG. 2, and a second data-processing programincluding program code for executing the processing in the flowchartshown in FIG. 3.

Although not specifically shown, the directory information includesdata, such as a recognition dictionary, associated with variousprograms. Programs for installing various programs on computers may alsobe included.

In the first embodiment, the accurate and efficient recognition ofmultiple pages of forms can be achieved without requiring a cumbersomeprocess, such as insertion of partition forms.

Moreover, the number of similarity calculations, and thus, the timerequired for recognition processing can be reduced by narrowing down thetarget forms to be processed on a page-by-page basis. In addition, theaccuracy of recognition can be improved, since recognition targets arenarrowed down by removing registered forms that are different from thesearch form.

Second Embodiment

A form recognition system according to a second embodiment of thepresent invention will now be described with reference to the drawings.

The second embodiment is executable by the form recognition systemhaving the same structure as in FIG. 1. In addition, the formrecognition system of the second embodiment is capable of identifyingforms with repeated appearance of specific pages.

Referring to FIG. 5, in a registered form ID2, an image data Eregistered on the second page and an image data F registered on thethird page repeatedly appear. That is, after the appearance of an imagedata D on the first page, the image data E and the image data Frepeatedly appear in such a manner as E, F, E, F, . . . E, and F. Then,an image data G on the fourth page appears at the end. In the tableshown in FIG. 5, the image data E on the second page is marked with arightward arrow, while the image data F on the third page is marked witha leftward arrow. The rightward arrow and the leftward arrow indicatethe start page of repetition and the end page of repetition,respectively.

If a range of three or more pages repeatedly appear, the rightward andleftward arrows indicate the start and end pages of repetition,accordingly, in the same manner as those described above. Moreover, aplurality of repetition ranges can be defined within a single form.

In the case where the start position and the end position are on thesame page, that is, the image data E on the second page is repeated insuch a manner as E, E, . . . , and E, the second page is marked withboth rightward and leftward arrows, or marked in other ways to definethe repetition.

In the second embodiment, such page repetition is taken intoconsideration in calculating the similarity value in step S307 in FIG.3. For example, if the similarity to the image data on the third page ofthe form ID2 is calculated and determined to be equal to or above apredetermined value in step S308, the similarities to both the imagedata G on the fourth page and to the image data E on the second pageappearing as a result of repetition are calculated in determining thesimilarity to the next page. Moreover, if another registered form stillremains as a recognition target, the similarity to that registered formis also to be calculated.

After the evaluation of similarity to the image data on the third page,if the similarity to the image data E is calculated in step S307 anddetermined to be equal to or above a predetermined value in step S308,and remains as a recognition target, it can be determined that therepetition of the image data E has occurred. In this case, thesimilarity to the image data F is to be subsequently calculated. If itis determined that the similarity to the image data F is equal to orabove the predetermined value, both the image data G and the image dataE are to be the comparison targets again.

In step S314, all of the pages (including repeated pages) of featurevalues are deleted, as recognition results, from the buffer.

As described above, in the second embodiment, the recognition of formswith multiple pages, specifically with a variable number of pagesincluding partial repetition, can be performed by automaticallydetermining the presence of repetition.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed embodiments. On the contrary, the invention isintended to cover various modifications and equivalent arrangementsincluded within the spirit and scope of the appended claims. The scopeof the following claims is to be accorded the broadest interpretation soas to encompass all such modifications and equivalent structures andfunctions.

This application claims priority from Japanese Patent Application No.2004-088207 filed Mar. 25, 2004, which is hereby incorporated byreference herein.

1. A form recognition system for identifying which of a plurality ofpreviously registered forms corresponds to a search form, each of theregistered forms containing multiple pages, the search form containingmultiple page images, the form recognition system comprising: featureobtaining means for obtaining page feature values from each page imagecontained in the search form; similarity calculating means forcalculating, in a predetermined page order, page similarity valuerepresenting similarity between the page feature value obtained from oneof the page images contained in the search form and page feature valueof a corresponding page contained in one of the registered forms ofcandidates, wherein the page feature value contained in the registeredforms include at least one of size, position and number of table framesin the registered form; target-form selecting means for removing one ofthe registered forms from the candidates when the page similarity valuebetween the page feature value obtained from one of the page imagescontained in the search form and the page feature value of thecorresponding page contained in one of the registered forms is smallerthan a predetermined value; and form determining means for repeatedlyprocessing each page of the search form in the predetermined page orderusing the similarity calculating means and the target-form selectingmeans, and determining one of the registered forms corresponding to thesearch form from among the candidates not removed by the target-formselecting means.
 2. The form recognition system according to claim 1,further comprising: feature-value registering means for storing the pagefeature values of the pages of the search form obtained by the featureobtaining means, wherein the similarity calculating means obtains, inthe predetermined page order, the page feature values of the search formfrom the feature-value registering means, and calculates the pagesimilarity value between the obtained page feature values of the searchform and the corresponding page feature values of the registered form.3. The form recognition system according to claim 1, wherein apossibility of a page repetition is stored in the registered form, andwherein the similarity calculating means calculates, if there is apossibility of page repetition in the search form, the similarity valuefor a page having a possibility of being repeated.
 4. The formrecognition system according to claim 1, wherein if the target-formselecting means excludes all of the registered forms from the candidatesbefore the form determining means determines one of the registered formscorresponding to the search form, the first page of the search form isdetermined to be unknown, and the next page is processed, as the toppage, by the similarity calculating means, the target-form selectingmeans, and the form determining means.
 5. The form recognition systemaccording to claim 1, wherein, if a plurality of registered forms remainas the candidates after the repeated processing, the form determiningmeans determines one of the registered forms corresponding to the searchform based on the page similarity values calculated by the similaritycalculating means.
 6. A form recognition method for controlling a systemfor identifying which of a plurality of previously registered formscorresponds to images of a search form with multiple pages, the formrecognition method comprising: using a processor to perform thefollowing: obtaining page feature values from the search form in apredetermined order; calculating a similarity value representingsimilarity between the page feature values obtained from the search formand page feature values of a registered form corresponding to the pagefeature values of the search form, wherein the page feature valuecontained in the registered forms include at least one of size, positionand number of table frames in the registered form; removing theregistered form from candidates for a recognition target if thesimilarity value is smaller than a predetermined value; and determininga form of recognition result, based on registered forms not removed inthe target-form selecting step, by repeatedly processing each page ofthe search form through obtaining page feature values, calculating thesimilarity value, and removing the registered form.
 7. The formrecognition method according to claim 6, further comprising: obtainingfeature values of each page of the search form, and storing the featurevalues in a feature-value registering memory; wherein in calculating thesimilarity value, the page feature values of the search form is obtainedfrom the feature-value registering memory, and the similarity valuebetween the obtained page feature values of the search form and thecorresponding page feature values of the registered form is calculated.8. The form recognition method according to claim 6, further comprising:storing a page repetition in the registered form; wherein in calculatingthe similarity value, if there is a possibility of page repetition inthe search form, the similarity of a page having a possibility of beingrepeated is calculated.
 9. The form recognition method according toclaim 6, wherein if, before completion of similarity calculation for allof the pages, all of the registered forms are excluded in removing theregistered form, the first page of the search form is determined to beunknown, and the next page is processed, as the top page, throughobtaining page feature values, calculating the similarity value,removing the registered form, and determining the form of recognitionresult.
 10. The form recognition method according to claim 6, wherein,if a plurality of registered forms remain as candidates, the form ofrecognition result is determined by determining the form of recognitionresult by using the similarity value to each page of each registeredform.
 11. A form recognition method for controlling a system foridentifying which of a plurality of previously registered formscorresponds to images of a search form with multiple pages, the formrecognition method comprising: using a processor to perform thefollowing: obtaining feature values of each page of the search form;calculating a similarity value representing similarity between thefeature values of each page of the search form, and the feature valuesof each page of the registered form, wherein the feature value containedin the registered forms include at least one of size, position andnumber of table frames in the registered form; and determining aregistered form corresponding to the search form, based on thesimilarity value calculated; wherein in calculating the similarityvalue, it is determined, based on a registered form memory for storing apossibility of page repetition in a registered form, whether or not aregistered form includes page repetition; and, for a registered formhaving the possibility of page repetition, the similarity to a pagehaving the possibility of page repetition is also calculated.
 12. Acomputer-readable storage medium storing a computer program for causinga computer to execute a form recognition method for controlling acomputer for identifying which of a plurality of previously registeredforms corresponds to images of a search form with multiple pages, theform recognition method causing the computer to perform stepscomprising: obtaining feature values of each page of the search form;calculating a similarity value representing similarity between thefeature values of each page of the search form, and the feature valuesof each page of the registered form, wherein the feature value containedin the registered forms include at least one of size, position andnumber of table frames in the registered form; and determining aregistered form corresponding to the search form, based on thesimilarity value calculated; wherein in calculating the similarityvalue, it is determined, based on a registered form memory for storing apossibility of page repetition in a registered form, whether or not aregistered form includes page repetition; and, for a registered formhaving the possibility of page repetition, the similarity to a pagehaving the possibility of page repetition is also calculated.
 13. Acomputer-readable storage medium storing a computer program for causinga computer to execute a form recognition method for controlling acomputer for identifying which of a plurality of previously registeredforms corresponds to images of a search form with multiple pages, theform recognition method causing the computer to perform stepscomprising: obtaining page feature values from the search form in apredetermined order; calculating a similarity value representingsimilarity between the page feature values obtained from the search formand page feature values of a registered form corresponding to the pagefeature values of the search form, wherein the rage feature valuescontained in the registered forms include at least one of size, positionand number of table frames in the registered form; removing theregistered form from candidates for a recognition target if thesimilarity value is smaller than a predetermined value; and determininga form of recognition result, based on registered forms not removed, byrepeatedly processing each page of the search form through obtainingpage feature values, calculating the similarity value, and removing theregistered form.
 14. The computer-readable storage medium according toclaim 13, further comprising: obtaining feature values of each page ofthe search form, and storing the feature values in a feature-valueregistering memory; wherein in calculating the similarity value, thepage feature values of the search form is obtained from thefeature-value registering memory, and the similarity value between theobtained page feature values of the search form and the correspondingpage feature values of the registered form is calculated.
 15. Thecomputer-readable storage medium according to claim 13, furthercomprising: storing a page repetition in the registered form; wherein incalculating the similarity value, if there is a possibility of pagerepetition in the search form, the similarity of a page having apossibility of being repeated is calculated.
 16. The computer-readablestorage medium according to claim 13, wherein if, before completion ofsimilarity calculation for all of the pages, all of the registered formsare excluded in removing the registered form, the first page of thesearch form is determined to be unknown, and the next page is processed,as the top page, through the obtaining page feature values, calculatingthe similarity value, removing the registered form, and determining theform of recognition result.
 17. The computer-readable storage mediumaccording to claim 13, wherein, if a plurality of registered formsremain as candidates, the form of recognition result is determined bydetermining the form of recognition result by using the similarity valueto each page of each registered form.